feat(playground): agent configuration and Skills Builder agent#2101
Open
feat(playground): agent configuration and Skills Builder agent#2101
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Extends the Playground with per-thread agent selection and a new built-in “Skills Builder” agent, including UI for managing agents and a specialized chat tool-result card for successful skill builds.
Changes:
- Added Agents management routes/pages (list, detail, create, edit) and an agent picker in chat settings.
- Updated chat runtime to resolve an agent per thread and run tool calls via
ToolLoopAgent, with optional agent-scoped built-in tool bundles (Skills). - Improved skills build/install UX (separate name/version prefill, consistent build titles, and a “Skill built” result card renderer).
Reviewed changes
Copilot reviewed 50 out of 50 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| renderer/src/routes/playground.agents.tsx | Adds /playground/agents route wrapper with sidebar + agents list page. |
| renderer/src/routes/playground.agents_.new.tsx | Adds create-agent route rendering the agent form in create mode. |
| renderer/src/routes/playground.agents_.$agentId.tsx | Adds agent detail route with sidebar and agent detail page. |
| renderer/src/routes/playground.agents_.$agentId_.edit.tsx | Adds edit-agent route with guards for missing/built-in agents. |
| renderer/src/route-tree.gen.ts | Registers new agents routes in the generated TanStack router tree. |
| renderer/src/features/agents/hooks/use-agents.ts | Introduces React Query hooks for listing/CRUD/duplicating agents and thread agent assignment. |
| renderer/src/features/agents/components/agents-page.tsx | Adds the Agents list UI with tabs and duplicate/create affordances. |
| renderer/src/features/agents/components/agent-detail-page.tsx | Adds agent detail UI (read-only for built-ins) with duplicate/edit/delete actions. |
| renderer/src/features/agents/components/agent-form-page.tsx | Adds create/edit agent form, including model picker and built-in tool bundle selection. |
| renderer/src/features/chat/components/agent-selector.tsx | Adds agent picker control to chat toolbar for selecting per-thread agent. |
| renderer/src/features/chat/components/chat-input-prompt.tsx | Injects AgentSelector into the chat prompt toolbar. |
| renderer/src/features/chat/components/chat-message.tsx | Adds build_skill success parsing and renders the “Skill built” card after tool output. |
| renderer/src/features/chat/components/tool-results/skill-build-result-card.tsx | Implements the “Skill built” card UI with Install/View details/Copy name actions. |
| renderer/src/features/chat/lib/parse-skill-build-result.ts | Adds parsing helper to detect successful build_skill outputs. |
| renderer/src/features/chat/components/playground-sidebar.tsx | Adds “Agents” navigation entry and refines sidebar spacing/styling. |
| renderer/src/features/chat/hooks/use-playground-threads.ts | Updates hook to accept nullable active thread id and adjusts active-thread IPC updates. |
| renderer/src/features/chat/components/model-selector.tsx | Refactors chat model selector to use the reusable ModelPicker. |
| renderer/src/features/chat/components/model-picker.tsx | Introduces shared model picker dropdown used by chat and agent form. |
| renderer/src/features/chat/components/mcp-server-selector.tsx | Updates MCP server selector button sizing/spacing to match updated toolbar. |
| renderer/src/features/chat/components/tests/playground-sidebar.test.tsx | Updates sidebar tests to mock router Link / useRouterState for new Agents nav. |
| renderer/src/features/chat/components/tests/agent-selector.test.tsx | Adds unit tests for the new agent selector behavior. |
| renderer/src/features/skills/components/dialog-install-skill.tsx | Adds defaultVersion prop and pre-fills the version field independently. |
| renderer/src/features/skills/components/card-build.tsx | Uses shared build title helper; updates install dialog prefill to name+version fields. |
| renderer/src/features/skills/components/build-detail-page.tsx | Uses shared build title helper; updates install dialog prefill to name+version fields. |
| renderer/src/features/skills/components/tests/card-build.test.tsx | Updates test expectations for split name/version prefill in install dialog. |
| renderer/src/features/skills/components/skill-detail-layout.tsx | Adds historyBack option to control back behavior for detail pages. |
| renderer/src/features/skills/lib/build-reference.ts | Adds shared getBuildTitle helper (trim + fallback). |
| preload/src/api/chat.ts | Exposes agents IPC API and adds agentId/thread.agentId fields to chat IPC types. |
| main/src/ipc-handlers/chat/index.ts | Registers new chat agents IPC handlers. |
| main/src/ipc-handlers/chat/agents.ts | Adds IPC handlers for listing/getting/CRUD/duplicating agents and thread assignment. |
| main/src/ipc-handlers/chat/tests/index.test.ts | Updates chat IPC registration test to include agents registration. |
| main/src/db/migrator.ts | Adds migration 005 for agents. |
| main/src/db/migrations/005-agents.ts | Creates agents table and adds threads.agent_id column. |
| main/src/db/tests/test-helpers.ts | Applies migration 005 in in-memory DB helper. |
| main/src/db/readers/threads-reader.ts | Hydrates agentId from threads.agent_id. |
| main/src/db/readers/agents-reader.ts | Adds DB readers for agents and per-thread agent id. |
| main/src/db/writers/threads-writer.ts | Persists agent_id on thread writes, preserving previous value when not provided. |
| main/src/db/writers/agents-writer.ts | Adds DB writers for agents and thread agent id. |
| main/src/chat/types.ts | Adds optional agentId to chat requests. |
| main/src/chat/threads-storage.ts | Adds optional per-thread agentId to thread metadata. |
| main/src/chat/streaming.ts | Switches streaming to ToolLoopAgent and resolves agent + built-in tool bundles per request/thread. |
| main/src/chat/agents/types.ts | Introduces agent config types, built-in ids, bundles, and defaults. |
| main/src/chat/agents/registry.ts | Adds agent registry (seed built-ins, CRUD, duplicate, thread assignment, resolution). |
| main/src/chat/agents/builtin-prompts.ts | Adds built-in prompts for ToolHive Assistant and Skills Builder. |
| main/src/chat/agents/builtin-agent-tools/index.ts | Adds built-in tool bundle factory/cleanup handle. |
| main/src/chat/agents/builtin-agent-tools/skills.ts | Implements Skills Builder tools (write_skill_files, build_skill) and cleanup. |
| main/src/chat/agents/tests/registry.test.ts | Adds unit tests for agent registry behavior and thread resolution. |
| main/src/chat/tests/streaming.test.ts | Adds unit tests for streaming agent resolution + tool bundle wiring. |
| main/src/app-events/when-ready.ts | Seeds built-in agents at app startup after migrations/reconcile. |
| main/src/app-events/tests/when-ready.test.ts | Updates startup test to mock agents seeding. |
Replaces the hardcoded ToolHive Assistant prompt with a per-thread agent resolved through an ai-sdk ToolLoopAgent. Agents are persisted in a new 'agents' table (migration 005) with a discriminated 'kind' ('builtin' | 'custom'), seed three built-ins on first launch (ToolHive Assistant, Skills Builder, Planner), and expose a small CRUD IPC surface (list, get, create, update, duplicate, delete) via 'preload/src/api/chat.ts'.
Threads now carry an optional 'agent_id' (migration + readers + writers) so each chat can pick its own agent and fall back to ToolHive Assistant when unset. Streaming ('main/src/chat/streaming.ts') looks up the thread's agent, composes its instructions as the system prompt, and binds any built-in tool bundle declared on the agent ('builtinToolsKey'). The bundle factory ('main/src/chat/agents/builtin-agent-tools') currently exposes 'skills' (write_skill_files + build_skill) and a placeholder 'planner'.
Tests: new registry CRUD coverage ('main/src/chat/agents/__tests__/registry.test.ts'), new streaming test for agent resolution + system prompt composition, updated IPC index + when-ready tests.
Splits the model dropdown into a presentational 'ModelPicker' ('renderer/src/features/chat/components/model-picker.tsx') and keeps 'ModelSelector' as a thin wrapper that owns the chat-settings-specific glue (provider credential lookup, onOpenSettings wiring). The picker now accepts a plain 'value / onChange / onClear / onOpenSettings' interface so it can be reused outside the chat (agent form, future screens) without duplicating the grouped-by-provider layout and search filter.
Behavior unchanged: same grouping, same filter, same empty state, same provider credential gating.
Adds a full agents feature under '/playground/agents' with a list page, a detail page (system prompt rendered as markdown on the right, metadata on the left — same layout as the skill detail), and a dedicated form page for create and edit. Built-in agents are read-only and offer a 'Duplicate' action instead; custom agents can be renamed, have their instructions edited in either raw textarea or Streamdown-rendered preview mode, and opt into a built-in tool bundle via a picker that hides empty bundles (e.g. Planner) from the UI.
Routes are flattened with trailing underscores ('playground.agents_.$agentId.tsx', 'playground.agents_.$agentId_.edit.tsx') so the detail and edit pages render under '/playground' — which owns the '<Outlet />' — rather than trying to nest inside the detail route. 'renderer/src/route-tree.gen.ts' is regenerated accordingly.
Also adds an optional 'historyBack' prop to 'SkillDetailLayout' ('renderer/src/features/skills/components/skill-detail-layout.tsx', defaults to 'true' for backwards compatibility) so the agent detail can force the Back button to navigate to '/playground/agents' instead of popping browser history.
Adds an 'AgentSelector' to the composer ('chat-input-prompt.tsx'), placed first in the row so the spacing stays tight even when the container narrows, and tightens the MCP server selector / model selector sizing to match. Selecting an agent persists 'agentId' on the thread via 'use-playground-threads.ts'; new threads inherit the most recently used agent and fall back to 'ToolHive Assistant' when no selection has been made.
Also adds an 'Agents' link to the playground sidebar ('playground-sidebar.tsx') so the management page is reachable alongside 'New chat', 'Starred', and 'Recents'. Icons for 'New chat' and the agent picker are now vertically aligned so the sidebar rows don't jitter when the container is narrow.
Covered by new 'agent-selector.test.tsx' and updated 'playground-sidebar.test.tsx'.
The install dialog used to accept a single 'defaultReference' string which callers derived by concatenating 'name:version', producing invalid OCI references like 'github-release-notes@v0.0.4' and leaking a version-only value into the Name field. The dialog now takes 'defaultReference' (bare name) and 'defaultVersion' (tag) separately so each form field is pre-filled correctly. Introduces a shared 'getBuildTitle' helper in 'renderer/src/features/skills/lib/build-reference.ts' and updates the local-build card and detail page to use it plus the new prop shape. Test coverage in 'card-build.test.tsx' is updated to assert the split values.
When the Skills Builder agent calls 'build_skill', the chat now renders a dedicated 'SkillBuildResultCard' right after the tool call instead of the raw JSON view. The card shows the skill name, description, version, tag, and a shortened digest with a tooltip, and exposes three actions: Install (opens the existing install dialog pre-filled with the bare Name and Version), View details (deep-links to the local build page), and Copy name (copies just the bare skill name). Parsing lives in 'renderer/src/features/chat/lib/parse-skill-build-result.ts' and gracefully falls back to the generic tool renderer when the 'build_skill' output doesn't match the expected shape, so malformed or older tool responses don't break the chat.
The previous 'feat(playground): add agents management pages' commit renamed 'playground.agents.$agentId.tsx' -> 'playground.agents_.$agentId.tsx' and added 'playground.agents_.$agentId_.edit.tsx' but the two renamed files never made it into the index, so clicking 'View details' or 'Edit' on a custom agent rendered nothing. Add them so the flattened routes resolve under '/playground' and actually mount.
The card used 'build.tag' verbatim, which is only a valid route param
when it matches a LocalBuild row in 'getApiV1BetaSkillsBuilds'. When the
list lookup in 'build_skill' lagged and we fell back to the user's
short input ('v0.0.4'), the navigate call hit /skills/builds/v0.0.4 and
404ed.
- Parser now preserves 'apiReference' and drops the 'reference' ('name:tag')
fallback from 'build.tag' so the field only holds a real registry tag.
- Card navigates to 'apiReference' first and only falls back to 'build.tag'
when it looks registry-prefixed (contains '/' or ':'); 'View details' is
hidden otherwise instead of producing a dead link.
- 'build_skill' tool seeds the fallback 'LocalBuild.tag' with 'apiReference'
instead of the user-supplied short tag, so future builds always emit a
navigable tag even if the listing endpoint lags.
'DialogInstallSkill' was prefilled with 'version ?? tag', so a build
without an explicit version ended up with the full OCI reference
('localhost/my-skill:v1.0.0') dumped into the Version field. Drop the
'tag' fallback — if no version is set, leave the field empty so it
defaults to 'latest' on install.
Planner was instructions-only and not really useful in practice — it was the sole reason the agent form needed a 'hasTools' filter on the tool bundle picker. Remove the agent, its prompt, its reserved 'BuiltinToolsKey' variant, and the now-unused 'hasTools' field; simplify the picker accordingly. 'seedBuiltinAgents' now deletes any leftover 'builtin.planner' row via a new 'LEGACY_BUILTIN_AGENT_IDS' list so existing dev databases migrate cleanly on startup.
'knip' flagged seven unused exports across the agents + model picker work. Scope them to their own modules — no functional change.
1e016b4 to
be08bed
Compare
…d agents flag gating
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the Playground with per-agent configuration and ships a new Skills Builder built-in agent that can scaffold and build an MCP skill end-to-end from chat. The new UI surfaces are gated behind a DevTools-only
agentsfeature flag so external users see the same chat as before while internal QA can flip it on.What's new
agentsfeature flag (DevTools-only, default off): hides the Agents sidebar entry and the chat composer's agent picker. TheToolLoopAgentruntime stays active and falls through to the ToolHive Assistant prompt, so chat behaviour with the flag off matchesmain. Toggle withFeatureFlag.agents.enable()/.disable()in DevTools./playground/agents: list, detail, create, and edit pages. Built-in agents are read-only with a Duplicate action; custom agents support name, description, default model, instructions (raw textarea or markdown preview), and an optional built-in tool bundle.ToolLoopAgentresolved per thread. Each agent carries its own system prompt, optional default model, and optional built-in tool bundle; user-enabled MCP tools continue to layer on top. With no UI to assign an agent, threads keepagent_id = nulland resolve tobuiltin.toolhive-assistant(same prompt as before).write_skill_files→build_skill) that writes a skill into a temp workdir and builds it into a local OCI artifact via the ToolHive API. Its system prompt is tuned to keep post-build replies to 1–2 sentences so the install CTA stays visible.build_skillshowing name / version / tag / digest with Install, View details, and Copy name actions. Install dialog is split into separate Name and Version fields so local builds install cleanly.ModelPicker: the model dropdown used by chat settings is now shared with the agent form so both UIs behave identically.How to validate
Flag off (default — external user behaviour):
Flag on (
FeatureFlag.agents.enable()in DevTools):build_skillsucceeds, the "Skill built" card appears with Install (opens the dialog with Name + Version prefilled), View details (navigates to the build detail page), and Copy name.lateston submit.